Speech to text (STT) systems and methods that produce text output or transcription based on audio input are known in the art. To convert speech to text, STT systems use language models. A language model is created or improved by a process known in the art as training. Generally, training includes assigning (and recording in a language model) probabilities or other values to words or phrases identified in input text or training material. However, since the training material is often very different from the actual input provided to an STT system in operation, known STT systems and methods may often produce poor transcriptions of input audio input.